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Abstract 

We report on a search for B'^ f^'^fJ-^ and — >■ ii^ ii~ decays in pp collisions at ^/s = 1.96 TeV 
using 171 pb~^ of data collected by the CDF II experiment at the Fermilab Tevatron Collider. 
The decay rates of these rare processes are sensitive to contributions from physics beyond the 
Standard Model. One event survives all our selection requirements, consistent with the background 
expectation. We derive branching ratio limits of B{Bg — > fi^ < 5.8 x 10~^ and B{B^ — >■ 
< 1.5 X 10~^ at 90% confidence level. 

PACS numbers: 14.40.Nd, 13.20.He 
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The rare flavor-changing neutral current decay 5" fi^fi [1] is one of the most sen- 
sitive probes to physics beyond the Standard Model (SM) [2-6]. The decay has not been 
observed and is currently limited to B{B'^ l^^l^ ) < 2.0 x 10^^ at 90% confidence level 



(CL) jrl], while the SM prediction is (3.5 ± 0.9) x 10^^ 8]. The limit on the related branch- 
ing ratio, B{B^ 1^^ 1^ ) < 1-6 x 10^^ |9], is approximately 1000 times larger than its SM 
expectation. The B{B^^ —>■ fi^fi^) can be significantly enhanced in various supersymmet- 
ric (SUSY) extensions of the SM. Minimal supergravity models at large tan/? [3-5] predict 
B{Bg — » a~^u^) < (9(10^^) in regions of parameter space consistent with the observed muon 
g — 2 [lO| and also with the observed relic density of cold dark matter [ll]. 50(10) mod- 
els 0], which naturally accommodate neutrino masses, predict a branching ratio as large 
as 10~^ in regions of parameter space consistent with these same experimental constraints. 
i?-parity violating SUSY models can also accommodate B{B^ A*^/^ ) up to 10~^ jjj. 
Correspondingly, the B{B^ —>■ fi^fi^) can be enhanced by the same models. Even modest 
improvements to the experimental limits can significantly restrict the available parameter 
space of these models. 

We report on a search for 5° /^^yU^ and B^ A*^/^^ decays using the upgraded 
Collider Detector at Fermilab (CDF II) at the Tevatron pp collider. The CDF II detector 
consists of a magnetic spectrometer surrounded by calorimeters and muon chambers and is 
described in detail in Ref. A cylindrical drift chamber (COT ) pr ovides 96 measurement 
layers, organized into alternating axial and ±2° stereo superlayers 13[ , and a five-layer silicon 
microstrip detector (SVX II) provides precise tracking information near the beamline jl^ . 
These are immersed in a 1.4 T magnetic field and measure charged particle momenta in the 
plane transverse to the beamline, pt- Four layers of planar drift chambers (CMU) detect 
muons which penetrate the five absorption lengths of calorimeter steel jl^. Another four 
layers of planar drift chambers (CMP) instrument 0.6 m of steel outside the magnet return 
yoke 16]. The CMU and CMP chambers each provide coverage in the pseudo-rapidity range 
|?7| < 0.6, where f] = — ln(tan |) and is the angle of the track with respect to the beamline. 
The dataset reported here corresponds to an integrated luminosity of £ = 171 ±10 pb^^ It! ]. 

The data used in this analysis are selected by dimuon triggers. Muons are reconstructed 
as track stubs in the CMU chambers. Two well-separated stubs are required and each is 
matched to a track reconstructed online using COT axial information [18]. The matched 
tracks must have px > 1.5 GeV/c. A complete event reconstruction performed online 
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confirms the Pt and track-stub matching requirements. If the overlapping CMP chambers 
contain a confirming muon stub, the track is required to have > 3 GeV/c. The two 
tracks must originate from the same vertex, be oppositely charged, and have an opening 
angle inconsistent with a cosmic ray event. The invariant mass of the muon pair must 
satisfy M^+^- < 6 GeV/c^. Events in which neither muon is reconstructed with a CMP 
stub must additionally satisfy + Pt > ^ GeV/c and M^+^- > 2.7 GeV/c^. This set 
of triggers is used for all the data included here and events passing these requirements are 
recorded for further analysis. 

Our offline analysis begins by identifying the muon candidates and matching them to the 
trigger tracks using COT hit information. To avoid regions of rapidly changing trigger effi- 
ciency, we omit muons with pt < 2 GeV/c. To reduce backgrounds from fake muons, stricter 
track-stub matching requirements are made and the vector sum of the muon momenta must 
satisfy {p^^'^ \ > 6 GeV/c. To ensure good vertex resolution, stringent requirements are 
made on the number of SVX 11 hits associated with each track. Surviving events have 
the two muon tracks constrained to a common 3-D vertex satisfying vertex quality require- 
ments. The two-dimensional decay length, \Lt\, is calculated as the transverse distance from 
the beamline to the dimuon vertex and is signed relative to p^ ^ . For each 5-candidate 
we estimate the proper decay length using A = cM^+^- ^ |. In the data, 2981 
events survive all the above trigger and offline reconstruction requirements. This forms a 
background-dominated sample with contributions from two principal sources: combinatoric 
background events with a fake muon and events from generic i?-hadron decays (e.g. se- 
quential semi-leptonic decays b cfi~X jX^ji'X or double semi-leptonic decay in gluon 
splitting events g bb fi~^fi~X). 

We model the signal decays using the Pythia Monte Carlo (MC) 'l^ tuned to inclusive 
S-hadron data [20]. The Pythia events are passed through a full detector simulation and 
satisfy the same requirements as data. To normalize to experimentally determined cross- 
sections, we require > 6 GeV/c and rapidity \y\ < 1. 

To discriminate B^(^^^ —>■ fJ'^f^'^ decays from background events we use these four variables: 
the invariant mass of the muon pair (M^+^-); the _B-candidate proper decay length (A); 
the opening angle (A$) between the 5-hadron flight direction (estimated as the vector 
Pt^'^ ) and the vector Lj^; and the i?-candidate track isolation (/) 21 1. Figure ^ shows the 
distributions of these variables for background-dominated data and MC signal events. 
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FIG. 1: Arbitrarily normalized distributions of the discriminating variables for events in our 
background-dominated data sample (solid) compared to Monte Carlo B'^ events (dashed). 



A "blind" analysis technique is used to determine the optimal selection criteria for 
these four variables. The data in the search window 5.169 < M^+^- < 5.469 GeV/c^ 
are hidden and the optimization performed using only data in the sideband regions, 
4.669 < M^+^- < 5.169 GeV/c^ and 5.469 < M^+^- < 5.969 GeV/c^. The search region 
corresponds to approximately ±4 times the two-track invariant mass resolution centered on 
the and masses We use the set of (M^+^-, A, A$, /) criteria which minimizes 
the a priori expected 90% CL upper limit on the branching ratio. For a given number of 
observed events, n, and an expected background of ribg, the branching ratio is determined 
using: 



0(5° 



< 



2 a BO C a etotai 

where N{n,ni,g) is the number of candidate B^ fi^fi" decays at 90% CL, estimated using 
the Bayesian approach of Ref. [2^ and incorporating the uncertainties into the limit. The a 
priori expected limit is given by the sum over all possible observations, n, weighted by the 
corresponding Poisson probability when expecting ribg. The B^ production cross-section is 
estimated as cr^o = j^aB+, where j- = Q] and 0"^+ is taken from Ref. j^]. For the 



B^i /i"*"/! limit we substitute a^o for aso, fd for fs, and assume fd = fu- The factor of two 
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in the denominator accounts for the charge-conjugate S-hadron final states. The expected 
background, nbg, and the total acceptance times efiiciency, aetotai, are estimated separately 
for each combination of requirements. 

For both signal and background, the variables A and A$ are the only correlated variables 
with a linear correlation coefficient of —0.3. Thus we estimate the number of background 
events as = ns\,{X, A$) // /m, where nsb(A, A$) is the number of sideband events passing 
a particular set of A and A$ cuts, // is the fraction of background events that survive a 
given / requirement, and /m is the ratio of the number of events in the search window to 
the number of events in the sideband regions. Since M^+^- and / are uncorrclatcd with the 
rest of the variables, we evaluate /m and // on samples with no A or A$ requirement, thus 
reducing their associated uncertainty. 

We estimate // from the background-dominated sample for a variety of thresholds. We 
investigate sources of systematic bias by calculating // in bins of M^+^- and A and conser- 
vatively assign a relative systematic uncertainty of ±5%. Since the M^+^- distribution of 
the background-dominated sample is well described by a first-order polynomial, fu is given 
by the ratio of widths of the search to sideband regions. 

MC studies demonstrate that our estimate of ribg accurately accounts for generic bb con- 
tributions, while two-body decays of S-mesons (-B^^^^) — > hf^h' , where — or K^) are 
estimated to contribute to the search region at levels at least 100 times smaller than our 
expected sensitivity. 

Using these background-dominated control samples, fi'^fi^ events and fi^fi'' events with 
A < 0, we compare our background predictions to the number of events observed in the search 
window for a wide range of (A, A<l>, /) requirements. No statistically significant discrepancies 
are observed. For example, using the optimized set of selection criteria described below and 
summing over these control samples, we get a total prediction of 3±1 events and observe five. 
Another cross-check is performed using a fake muon enhanced /i^/i" sample. By requiring 
at least one of the muon legs to fail the muon identification requirements, we reduce the 
signal efficiency by a factor of 50 while increasing the background acceptance by a factor of 
three. In this sample, using the optimized requirements, we predict 6 ± 1 and observe seven 
events. 

We estimate the total acceptance times efficiency as a etotai — o; etrig Creco Cfinab where a 
is the geometric and kinematic acceptance of the trigger, etrig is the trigger efficiency for 
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events in the acceptance, ej-eco is the offline reconstruction efficiency for events passing the 
trigger, and efinai is the efficiency for passing the final cuts on the discriminating variables 
for events satisfying the trigger and reconstruction requirements. For the optimization, only 
Cfinai changes as we vary the requirements on M^+^-, A, A<l>, and /. 

The acceptance is estimated as the fraction of S^^^^ A*"*"/^" MC events which fall 
within the geometric acceptance and satisfy the kinematic requirements of at least one of 
the analysis triggers. We find a — (6.6 ± 0.5)%. The uncertainty includes roughly equal 
contributions from systematic variations of the modehng of the 5-hadron pr spectrum and 
longitudinal beam profile, and from the statistics of the sample. It also includes negligible 
contributions from variations of the beamline offsets and of the detector material description 
used in the simulation. 

The trigger efficiency, including the effects of the offline-to-trigger track matching, is 
estimated from samples of J/'^ — > decays selected with a trigger requiring only one 

identified muon. The data are used to parameterize the trigger efficiency as a function of 
Pt and rj for the unbiased muon. The efficiency for -8°^^-^ fi^fi^ decays is determined 
by the convolution of this parameterization with the {pi^ , f]'^^ , pj. ,77^ ) spectra of signal 
MC events within the acceptance. Including the online reconstruction requirements, the 
trigger efficiency is etrig = (85 ±3)%. The uncertainty is dominated by the systematic 
uncertainty accounting for kinematic differences between J/ip ^ A*"*"/^" and -^^(d) l^'^l-i'^ 
decays and also includes contributions from variations in the functional form used in the 
parameterization, the effects of two-track correlations, and sample statistics. 

The offline reconstruction efficiency is given by the product ereco = ^cot esvx, where 
ecoT is the absolute reconstruction efficiency of the COT, e^u is the muon reconstruction 
efficiency given a COT track, and esvx is the fraction of reconstructed muons which satisfy 
the SVX II requirements. Each term is a two-track efficiency. A hybrid data-MC method 
is used to determine ecoT- Occupancy effects are accounted for by embedding COT hits 
from MC tracks in data events. The MC simulation is tuned at the hit level to reproduce 
residuals, hit width and hit usage in the data. For embedded muons with px > 2 GeV/c, 
we measure ecoT = 99%. Using the unbiased J/ip ^ fJ'^fJ'' samples, we estimate the muon 
reconstruction efficiency, including the track-stub matching requirements, to be 96%. A 
sample of J/'^ — > events satisfying our COT and muon reconstruction requirements 

is used to determine esvx = 75%. The total reconstruction efficiency is given by the above 



11 



product, ereco = (71 ± 3)%. The uncertainty is dominated by the systematic uncertainty 
accounting for kinematic differences between J/ip ^ /^^/^~ and -B^^^-j — > /U^/i^ decays and 
also includes contributions from the variation of the COT simulation parameters and sample 
statistics. 

The efficiency efinai is determined from the B^s{d) ~^ f^~^f^~ sample and varies from 
28 — 78% over the range of (M^+^-, A, A$, /) requirements considered in the optimization. 
The MC modeling is checked by comparing the mass resolution and A, A$, and / efficiency 
as a function of selection threshold for J/iIjK^{J/iIj events. The — >■ 

J/ipK^ MC sample is produced in the same manner as the i?^ f^^f^^ sample. The 
B^ J/ipK^ data sample is collected using dimuon triggers very similar to those used in 
the analysis, but with a larger acceptance for B^ J/ipK^ decays. We make the same 
requirements on the dimuon tracks and vertex as employed in the analysis. The MC efficiency 
is consistent with the sideband-subtracted data efficiency for a range of cut thresholds within 
5% (relative), which is assigned as a systematic uncertainty on efinai- In both the data and 
the MC samples the mean of the three-track invariant mass distribution is within 3 MeV/c^ 
of the world average B^ mass. The two-track invariant mass resolution is well described by 
the MC. 

The optimal set of selection criteria uses a ±80 MeV/c^ search window around the B^ 
mass, A > 200 /im, A$ < 0.10 rad and / > 0.65. The mass resolution, estimated from the 
MC for the events surviving all requirements, is 27 MeV/c^ so that the B^ and masses 
are resolved. We define a separate search window centered on the world average B^ mass 
and use the same set of selection criteria for the B^ fi~^fi~ search. The total acceptance 
times efficiency is aetotai = (2.0 ± 0.2)% for both decays. 

Using these criteria one event survives all requirements and has an invariant mass of 
= 5.295 GeV/c^, thus falling into both the B^ and B^ search windows as shown in 
Figure 121 This is consistent with the 1.1 ± 0.3 background events expected in each of the 
5° and 5° mass windows. We derive 90% (95%) CL hmits of i3(5° ^i+fi') < 5.8 x 10^^ 
(7.5 X 10"^) and i3(S^ ^ /i+/i~) < 1.5 x 10^^ (1.9 x 10"^). The new 5° fi+fi' limit 
improves the previous limit |3] by a factor of three and significantly reduces the allowed 
parameter space of i?-parity violating and 50(10) SUSY models jj, 0]. The B^ 
limit is slightly better than the recent limit from the Belle Collaboration j^. 
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FIG. 2: The /i'''/^ invariant mass distribution of the events in the sideband and search regions 
satisfying all requirements. 
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